Enabling dynamic and intelligent workflows for HPC, data analytics, and AI convergence

The evolution of High-Performance Computing (HPC) platforms enables the design and execution of progressively larger and more complex workflow applications in these systems. The complexity comes not only from the number of elements that compose the workflows but also from the type of computations they perform. While traditional HPC workflows target simulations and modelling of physical phenomena, current needs require in addition data analytics (DA) and artificial intelligence (AI) tasks. However, the development of these workflows is hampered by the lack of proper programming models and environments that support the integration of HPC, DA, and AI, as well as the lack of tools to easily deploy and execute the workflows in HPC systems. To progress in this direction, this paper presents use cases where complex workflows are required and investigates the main issues to be addressed for the HPC/DA/AI convergence. Based on this study, the paper identifies the challenges of a new workflow platform to manage complex workflows. Finally, it proposes a development approach for such a workflow platform addressing these challenges in two directions: first, by defining a software stack that provides the functionalities to manage these complex workflows; and second, by proposing the HPC Workflow as a Service (HPCWaaS) paradigm, which leverages the software stack to facilitate the reusability of complex workflows in federated HPC infrastructures. Proposals presented in this work are subject to study and development as part of the EuroHPC eFlows4HPC project.This work has received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 955558. The JU receives support from the European Union’s Horizon 2020 research and innovation programme and Spain, Germany, France, Italy, Poland, Switzerland and Norway. In Spain, it has received complementary funding from MCIN/AEI/10.13039/501100011033, Spain and the European Union NextGenerationEU/PRTR (contracts PCI2021-121957, PCI2021-121931, PCI2021-121944, and PCI2021-121927). In Germany, it has received complementary funding from the German Federal Ministry of Education and Research (contracts 16HPC016K, 6GPC016K, 16HPC017 and 16HPC018). In France, it has received financial support from Caisse des dépôts et consignations (CDC) under the action PIA ADEIP (project Calculateurs). In Italy, it has been preliminary approved for complimentary funding by Ministero dello Sviluppo Economico (MiSE) (ref. project prop. 2659). In Norway, it has received complementary funding from the Norwegian Research Council, Norway under project number 323825. In Switzerland, it has been preliminary approved for complimentary funding by the State Secretariat for Education, Research, and Innovation (SERI), Norway. In Poland, it is partially supported by the National Centre for Research and Development under decision DWM/EuroHPCJU/4/2021. The authors also acknowledge financial support by MCIN/AEI /10.13039/501100011033, Spain through the “Severo Ochoa Programme for Centres of Excellence in R&D” under Grant CEX2018-000797-S, the Spanish Government, Spain (contract PID2019-107255 GB) and by Generalitat de Catalunya, Spain (contract 2017-SGR-01414). Anna Queralt is a Serra Húnter Fellow.With funding from the Spanish government through the ‘Severo Ochoa Centre of Excellence’ accreditation (CEX2018-000797-S)

Altimetry for the future: Building on 25 years of progress

In 2018 we celebrated 25 years of development of radar altimetry, and the progress achieved by this methodology in the fields of global and coastal oceanography, hydrology, geodesy and cryospheric sciences. Many symbolic major events have celebrated these developments, e.g., in Venice, Italy, the 15th (2006) and 20th (2012) years of progress and more recently, in 2018, in Ponta Delgada, Portugal, 25 Years of Progress in Radar Altimetry. On this latter occasion it was decided to collect contributions of scientists, engineers and managers involved in the worldwide altimetry community to depict the state of altimetry and propose recommendations for the altimetry of the future. This paper summarizes contributions and recommendations that were collected and provides guidance for future mission design, research activities, and sustainable operational radar altimetry data exploitation. Recommendations provided are fundamental for optimizing further scientific and operational advances of oceanographic observations by altimetry, including requirements for spatial and temporal resolution of altimetric measurements, their accuracy and continuity. There are also new challenges and new openings mentioned in the paper that are particularly crucial for observations at higher latitudes, for coastal oceanography, for cryospheric studies and for hydrology. The paper starts with a general introduction followed by a section on Earth System Science including Ocean Dynamics, Sea Level, the Coastal Ocean, Hydrology, the Cryosphere and Polar Oceans and the ‘‘Green” Ocean, extending the frontier from biogeochemistry to marine ecology. Applications are described in a subsequent section, which covers Operational Oceanography, Weather, Hurricane Wave and Wind Forecasting, Climate projection. Instruments’ development and satellite missions’ evolutions are described in a fourth section. A fifth section covers the key observations that altimeters provide and their potential complements, from other Earth observation measurements to in situ data. Section 6 identifies the data and methods and provides some accuracy and resolution requirements for the wet tropospheric correction, the orbit and other geodetic requirements, the Mean Sea Surface, Geoid and Mean Dynamic Topography, Calibration and Validation, data accuracy, data access and handling (including the DUACS system). Section 7 brings a transversal view on scales, integration, artificial intelligence, and capacity building (education and training). Section 8 reviews the programmatic issues followed by a conclusion

Altimetry for the future: building on 25 years of progress

In 2018 we celebrated 25 years of development of radar altimetry, and the progress achieved by this methodology in the fields of global and coastal oceanography, hydrology, geodesy and cryospheric sciences. Many symbolic major events have celebrated these developments, e.g., in Venice, Italy, the 15th (2006) and 20th (2012) years of progress and more recently, in 2018, in Ponta Delgada, Portugal, 25 Years of Progress in Radar Altimetry. On this latter occasion it was decided to collect contributions of scientists, engineers and managers involved in the worldwide altimetry community to depict the state of altimetry and propose recommendations for the altimetry of the future. This paper summarizes contributions and recommendations that were collected and provides guidance for future mission design, research activities, and sustainable operational radar altimetry data exploitation. Recommendations provided are fundamental for optimizing further scientific and operational advances of oceanographic observations by altimetry, including requirements for spatial and temporal resolution of altimetric measurements, their accuracy and continuity. There are also new challenges and new openings mentioned in the paper that are particularly crucial for observations at higher latitudes, for coastal oceanography, for cryospheric studies and for hydrology. The paper starts with a general introduction followed by a section on Earth System Science including Ocean Dynamics, Sea Level, the Coastal Ocean, Hydrology, the Cryosphere and Polar Oceans and the “Green” Ocean, extending the frontier from biogeochemistry to marine ecology. Applications are described in a subsequent section, which covers Operational Oceanography, Weather, Hurricane Wave and Wind Forecasting, Climate projection. Instruments’ development and satellite missions’ evolutions are described in a fourth section. A fifth section covers the key observations that altimeters provide and their potential complements, from other Earth observation measurements to in situ data. Section 6 identifies the data and methods and provides some accuracy and resolution requirements for the wet tropospheric correction, the orbit and other geodetic requirements, the Mean Sea Surface, Geoid and Mean Dynamic Topography, Calibration and Validation, data accuracy, data access and handling (including the DUACS system). Section 7 brings a transversal view on scales, integration, artificial intelligence, and capacity building (education and training). Section 8 reviews the programmatic issues followed by a conclusion

Extension orientée objet d'un SGBD relationnel

The objective of this work is the specification and development of a Relational Database Management System (RDBMS) which incorporates "Object" technology and concepts. The principle of our approach is to extend relational domains with Abstract Data Types (ADT) and consists in loosely coupling object concepts and mechanisms to the relational model and system. This introduces new modelling and optimization problems requiring further study. First, the data model and the features of the extension are defined. The concept of Abstract Data Type is introduced in order to express new domains: an ADT specifies a data structure together with a set of methods (functions) which constitutes its manipulation interface. A simple inheritance mechanism is provided. Constructors are supplied to define the data structure of a type, introducing the notion of complex object. The concept of sharing, based on object identity, is an important contribution of this work. The language which materializes the model is a SQL extension, named ESQL; methods are presently written using a C extension. The realization of such a system consists in the implementation of different components providing the object support and in the integration of these components to an existing RDBMS kernel. There are three main modules. The type manager is a part of the relational catalogue manager which handles ADT definitions. The method manager provides different functions like compilation and execution. The object manager supplies object storage and manipulation capabilities; this part provided the opportunity to study advanced object storage techniques.L'objectif de ce travail est la conception et la réalisation d'un Système de Gestion de Base de Données Relationnel (SGBDR) intégrant les concepts et la technologie "objets". Le principe de notre approche est d'étendre les domaines relationnels aux types abstraits (ADT) et revient à coupler de façon relativement faible les concepts et mécanismes objets au modèle et à un système relationnels. Cela introduit des problèmes de modélisation et d'optimisation nouveaux qui restent à étudier. Dans un premier temps, le modèle de données et les caractéristiques de l'extension sont définis. La notion de type abstrait est introduite pour exprimer de nouveaux domaines : un ADT définit une structure de données et un ensemble de méthodes (fonctions) qui constituent son unique interface de manipulation. Un mécanisme d'héritage simple est offert. Des constructeurs sont disponibles pour définir la structure de données d'un type ; on introduit ainsi la notion d'objet complexe. Le concept de partage, associé à l'identité d'objet, est un apport important de ce travail. Le langage associé au modèle est une extension de SQL appelée ESQL ; le langage d'écriture des méthodes actuellement disponible est une extension de C. La mise en oeuvre d'un tel système consiste à développer les composants nécessaires au support d'objets et à les intégrer à un noyau de SGBDR existant. Elle permet de mettre en évidence trois modules principaux. Le gestionnaire de types est un complément du gestionnaire de catalogue relationnel qui gère les définitions d'ADT. Le gestionnaire de méthodes regroupe un ensemble de fonctions allant de la compilation à l'exécution. Le gestionnaire d'objets assure le stockage et la manipulation des objets complexes (instances d'ADT) ; cette partie à notamment permis d'étudier des techniques évoluées de stockage d'objets

Le Cerga : la télémétrie laser-Lune

International audienc

DORIS-class oscillator under radiations: The Jason family of satellites

International audienc

A framework for managing dynamic service-oriented component architectures

International audienceSoftware development is moving from monolithic to modular, dynamically composable applications. Modularity and dynamicity are the basis for software evolution since they provide the means of adapting and updating an application. Currently, service-oriented component models are one of the most advanced technologies for creating dynamic applications. These component models, which inherit concepts from both component-based software engineering and service oriented computing, provide a programming model that both supports and encourages dynamic reconfigurations. Although reconfigurations are possible, it is still difficult to manage a dynamic application's architecture, especially in highly dynamic environments. In this paper, we provide an overview of the benefits of service oriented component models and the main concepts used in their implementations. We provide a model that reifies important concepts and can be used to manage the application's architecture and its dynamic reconfigurations. Finally we propose a generic framework that allows for the creation of specialized architecture managers, capable of both monitoring and controlling dynamic service-oriented component applications

Extending a Relational DBMS to support Engineering Databases

International audienc

The Smithsonian Astrophysical Observatory (SAO) and the Centre National d’Études Spatiales (CNES): contributions to the international laser ranging network

International audienceThe Smithsonian Astrophysical Observatory (SAO) and the Centre National d’Études Spatiales (CNES) worked closely together in the early years of the space program to deploy satellite laser ranging (SLR) systems at overseas sites to enhance global coverage to support specific missions. The data were routinely made available for use by the science community for programs in geodesy, gravity field, atmospheric physics, and ultimately for oceanography and geodynamics. SAO and CNES organized campaigns for international participation and loaned each other equipment to enhance the network. SAO and CNES provided technical expertise and advice to a number of other groups as they planned and deployed SLR systems. In this paper, we will discuss the history and the role of the two institutions in the building of the international SLR network